Employee Sentiment (GlassDoor) & Stock Returns in the Oil Sector
Preface
Employees are the heart and soul of a business. Their actions and performance are what drive the profitability of a business and it’s ability to maximize shareholder value. In the following document, we look to address the following question:
Is there a relationship between employee sentiment and excess returns?
By building a webscraping program and applying it to GlassDoor, we were able to compile a comprehensive of employee reviews across a va of companies:
- ExxonMobil (XOM)
- Chevron (CVX)
- Shell (SHEL)
- BP (BP)
For context on the analysis that occurs below, here is a chart that represents their respective share prices from the last 10 years:
We wanted to select companies that all operate in the same industry and differ by market cap. This ensures that our analysis truly represents differentiating factors in relation to excess returns.
Firstly, we needed to make sense of all the reviews and how best to compile and analyze them. Natural Language processing is the computational analysis of language and speech. This essentially analyzes words and phrases and rates them numerically or categorically based off of the emotion and message they are conveying. Here is a short description of the two methods we applied:
- Bing: A dataset of 6,786 words with binary positive and negative sentiment scores
| word | sentiment |
|---|---|
| 2-faces | negative |
| abnormal | negative |
| abolish | negative |
| zest | positive |
| zippy | positive |
| zombie | negative |
- Afinn: A dataset of 2477 words with scores ranging from -5 to 5:
| word | value |
|---|---|
| breathtaking | 5 |
| superb | 5 |
| thrilled | 5 |
| amazing | 4 |
| awesome | 4 |
| brilliant | 4 |
We also utilized the star ratings that employees leave whenever a review is posted (ranging from 1 star to 5 stars). To organize our data into workable metrics. We used individual words instead of ngrams (collection of words or short sentences) as the bulk of the reviews are short and sometimes contain only one word. To summarize, we opted for the following:
- Grouped reviews into monthly, quarterly and annual bins.
- Bing was organized into percentage of reviews that were deemed positive and averaged over monthly, quarterly and annually.
- Afinn was organized into average total score over monthly, quarterly and annually.
- Star ratings were averaged over monthly, quarterly and annually.
Here are some visuals that are separated by company, depicting how these various metrics change through time. All charts use monthly data to help tell the story:
# A tibble: 6 × 4
date Company word value
<date> <chr> <chr> <dbl>
1 2018-01-09 bp terrible -3
2 2018-01-09 bp terrible -3
3 2018-01-15 bp negative -2
4 2018-01-10 bp good 3
5 2018-01-26 bp best 3
6 2018-01-10 bp great 3
Alpha Analysis
- Using CAPM, Alphas were calculated annually for each individual stock
- 1 Year Beta
- S&P500 utilized as a proxy for market return
- 10 Year U.S. Treasury security utilized as a proxy for the risk free rate
- Annual Alphas were regressed annually against the prior year’s average review ratings and written sentiment, as defined by the Affin dictionary
- Bing NLP excluded due to model multicollinearity
Rational
- Working with alpha instead of logarithmic returns captures returns that are not related to market-wide price movements
- Attempts to isolate returns to firm specific factors such as governance or commodity price exposure
- Utilize sentiment indicators generated from the prior years review as a proxy to capture possible alpha derived from strong employee satisfaction
- One year time frame captures returns and sentiment on a longer term horizon
- Attempt to better capture governance effects
Findings
Our sentiment proxy had no explanatory power when attempting to predict alpha returns in the proceeding year - Model fit was poor for all four companies - No model or factor significance was found - Counter-intuitive beta directionality
| Characteristic | Chevron | BP | ExxonMobil | Shell | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Beta | 95% CI1 | p-value | Beta | 95% CI1 | p-value | Beta | 95% CI1 | p-value | Beta | 95% CI1 | p-value | |
| mean_rating | -2.3 | -4.6, -0.10 | 0.043 | 0.91 | -1.2, 3.0 | 0.3 | -1.7 | -4.4, 0.94 | 0.2 | -1.2 | -4.1, 1.6 | 0.3 |
| sentiment | 1.5 | 0.14, 2.8 | 0.036 | -0.23 | -2.0, 1.6 | 0.8 | 0.46 | -1.9, 2.8 | 0.7 | 1.5 | -1.5, 4.5 | 0.3 |
| 1 CI = Confidence Interval | ||||||||||||
Portfolio-based Analysis
Two portfolios of our selected companies were created:
- Portflio 1: Naive equally-weighted portfolio
- Rebalanced yearly based upon the prior years closing price
- Portfolio 2: Optimized portfolio utilizing three sentiment indicators
- Generated weights utilizing a combination the input indicators to create a single sentiment measure
- Optimization process to determine the ideal relative sentiment mix
- Training window: 2014-2021
- Rebalanced annually using an aggregation sentiment data from the prior year
- Sentiment is treated on a absolute basis as opposed to on a relative basis to prior years
- Companies are proportional de-weighted in cases of ‘review slippage’ relative to their peers
- No additionally compounds are included for +/- review drift
Rational
- Firms with consistently superior Glassdoor reviews, as measured by our sentiment indicators, will be receive more weight
- Shell maintained relatively superior reviews over the analysis window
- Attempts to capture excess returns generated by superior employee sentiment
- Naive equally weighted portfolio intends to serve as a baseline
- Captures returns on longer term horizon
Findings
Utilizing our sentiment indicators, we were not able to generate a portfolio that outperformed our baseline during the test period
- Assigning 72% weight to our Bing indicator and 28% weight our Afinn indicator (both NLP) yielded returns in the 99th percentile with variance in the 90th percentile (training)
- Test window (2020-2023) suggests superior returns observed in the training window were likely spurious
| Return | Variance | Positive Prop. | Rating | Sentiment | Portfolio |
|---|---|---|---|---|---|
| 0.4307042 | 0.0706839 | 0.72 | 0 | 0.28 | Optimized Sentiment-based |
| 0.4282957 | 0.0738970 | NA | NA | NA | Naive |